A novel contextual topic model for multi-document summarization

Information overload becomes a serious problem in the digital age. It negatively impacts understanding of useful information. How to alleviate this problem is the main concern of research on natural language processing, especially multi-document summarization. With the aim of seeking a new method to help justify the importance of similar sentences in multi-document summarizations, this study proposes a novel approach based on recent hierarchical Bayesian topic models. The proposed model incorporates the concepts of n-grams into hierarchically latent topics to capture the word dependencies that appear in the local context of a word. The quantitative and qualitative evaluation results show that this model has outperformed both hLDA and LDA in document modeling. In addition, the experimental results in practice demonstrate that our summarization system implementing this model can significantly improve the performance and make it comparable to the state-of-the-art summarization systems. While the rapid growth of the World Wide Web has resulted in bringing people from different parts of the world much closer and able to access to a vast amount of information on their fingertips, it has also created a serious problem of information overload, which impacts people negatively in comprehending useful information. How to alleviate this problem is of concern to the research on automatic text summarization, especially the extraction based multi-document summarization. The main task of the extraction based multi-document summarization is to extract the most important sentences from multiple documents and format them into a summary. Therefore, finding an appropriate method to justify the importance (or relevance) of a string of text (e.g., a sentence) dominates this research area. Many proposed approaches use statistical methods, lexical chains, graph-based algorithms, or Bayesian language models to produce summaries. For example, a well-known summarizer, SumBasic in statistical methods, specifies the importance of a sentence in a document by counting term frequency or inverse document frequency (TF-IDF) exclusive of stop words & Nenkova, 2007). Others identify the relevance of a sentence by using bigram pseudo sentences for using hybrid statistical sentence extraction (Ko & Seo, 2008), rhetoric-based multi-document summarization (Atkinson & Munoz, 2013) and semantic document concept technique (Ye, Chua, Kan, & Qiu, 2007) based on the rhetorical structure theory (RST) (Mann & Thompson, 1988) for analyzing grammatical structures in discourses. However, heavy reliance on human expert's rhetorical roles and linguistic knowledge bases is definitely a bottleneck for RST based approaches.